MR: Merge Worker issue #57 to main #63
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
For using look examples at the end of validator.py
For testing validator, models from https://www.entsoe.eu/Documents/CIM_documents/Grid_Model_CIM/TestConfigurations_packageCASv2.0.zip were used. Examples in the bottom of the file
…ip file location
…somewhat faster than printing the log to console (validator.py line 706)
…e files in directory, parses them to the form as they would have come from OPDE and continues as usual. Plus some bug fixes for running it externally
…troduces two handlers: HandlerGetModels (gets data from minio and feeds to cgm_compose, HandlerMergeModels calls the merge and posts the results
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Possible implementation for #57
Can do:
Get Context from task message from RabbitMQ: gets data, merges models based on properties fields, add context to logger (elastic logs)
Get Data from Object Storage: gets data from local storage (elastic, minio) fallback can be set to opdm (options to run the validation or model retriever pipeline)
Publish Data to Object Storage: publishes data to local storage, file tree (prefix) is composed from the filename. Metadata can be published to elastic
Cannot do:
Publish Data to OPDM: publishes the data to opdm but from the opdm side, validation still fails